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SIMULATION IN THE TRAINING OF R & D PROJECT MANAGERS 



by 

Duane H. 01 liman and Desmond L* Cook 

I. THE PROBLEM 

For the past two and one-half years the staff of the Educational 
Program Management Center at The Ohio State University has been Involved 
with the training of R & D program and project managers* As a part of 
the training program, there has always been some attention to the appli- 
cation of the techniques emphasized in the training program in order to 
see the degree of understanding of the concepts, techniques, and abilities 
of the participants. This application has taken the form of relatively 
simple practical exercises* 

Ouring the first series of four one-week training programs sponsored 
by the Research Training Branch at USOE and the 1967 AERA Presession, both 
undertaken by the Educational Program Management Center, the practical 
exercise used was one which the director of the Center developed primarily 
as an output of the PERT Project (l). This exercise Is considered to be 
"canned" In that It was highly structured in Its approach, did not allow 
for much freedom, and was too narrow In content since It was based on a 
survey of an hypothetical parking problem at a university. In addition, 
an evaluation of the above mentioned training program raised some question 
about the relevancy of the exercise problem to the situations encountered 
by many educational researchers* 

The authors wish to express their appreciation to John Church, Research 
Assistant, for his help with data analysis. 
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Out of this need for a more realistic and broader based exercise came 
the decision to develop an exercise using the technique of simulation. It 
was felt that Increased realism and greater involvement by participants 
would be obtained by the use of a simulated situation which demanded more 
complex skills, some degree of role playing, and increased opportunity for 
decision making in various education related management positions. 

During the past sixteen months, the staff of the EPMC has been Involved 
in the development and testing of a simulation exercise which Is designed 
to meet these needs. This paper describes the development and use of this 
exercise in relation to the training of R & D project managers. Before 
describing the development of the simulation, perhaps a brief definition 
of both nature of project management and simulation is important to set 
the context. 

PROJECT MANAGEMENT 

Projects are found in almost all existing educational organizations 
today as well as in every business, industry, and governmental level. 

They are "unique, well-defined efforts to produce certain specified results 
at a particular point In time, and typically cut across many functional and 
organizational lines (2)." 

Project management began In the 1950* s with the Navy Polaris Program 
which was "a complex program consisting of the development and production 
of the warhead, the missile launcher, and the submarine carrier through 
prime and sub-contractor structure (3)*" The use of project management 
tools of the Program Evaluation and Review Technique (PERT), system analysis 
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and similar applications have been credited with bringing the Polaris pro- 
ject to fruition a full two years ahead of schedule. As a result of such 
success, project management emerged as an Integral management force through- 
out DOD, NASA, and the aerospace industry, as well as having literally 
hundreds of applications In business and industry In this country and 
abroad. Applications have been made of project management to non-defense 
and non- techno log I cal situations at a much slower pace. Applications to 
education and the social systems, however, have received some attention In 
recent years (4). 

Important to our consideration here Is the quest lor^, "Can training In 
project management techniques provide educational project managers with 
the tools to Implement some valuable project earlier and thereby have an 
untold positive effect on educational practices?" Hope for even a modest 
improvement has led to the development of the Educational Program Management 
Center, which intends to undertake research, development, and training in 
the application of management techniques to education. 

SIMULATION 

The first modern use of simulation was the development of the business 
game by the American Management Association In 1956, although the study and 
considerations given military war games as a basis for this is well docu- 
mented (5). Since that time the developments In simulation have been ex** - 
tremely rapid and varied as the result of the influence of electronic com- 
puters, the development of the theory of games In operations research, and 
the spread of simulation to almost every discipline. The Importance of 



simulation in educational research and research related areas Is seen by 
the formation of such groups as a Special Interest Group in AERA concerned 
with instructional simulation. Leaders in this movement Include Donald 
Cruickshank (6) as Chairman and Paul Twelker (7) as secretary. The growth 
of this group to over 40 persons from February through October of 1968 
indicates the interest of research-oriented educators in simulation. The 
range of definitions and uses of simulation extends from computerized 
mathematical modeling to election forecasts, from space flight simulation 
and training of pilots and astronauts to in-basket techniques for training 
school administrators (8), As used in this study simulation refers to a 
decision-making exercise used as part of a training program in understanding 
and applying a relatively large number of management processes and concepts 
important in the areas of educational program and project management, 

1 1 , DEVELOPMENT AND USE OF SIMULATION MATERIALS 

The need for a broader based exercise, particularly for use in conjunction 
with the training activities of the Educational Program Management Center, 
was established earlier in the paper. Following the decision to use simu- 
lation as an attempt to fill that need, a study of simulation was undertaken. 

It was found that even with the availability of an excess of a thousand 
references on the topic of simulation, very little of the literature presented 
practical techniques or guidelines which could be employed in the design of 
e simulation exercise. (9), Consequently, the authors had to proceed largely on 
the basis of their own experience and that of others who had been involved 
In actual simulation exercises. 



The Initial decisions regarding the design and development of the 
simulation materials include the following items: 

(1) The materials would first be used to train educational 
researchers or those dealing with research and should 
therefore involve one or more problems or projects with 
which such people might be familiar. 

(2) Although the immediate problem would probably deal with 
the management of research (a specific project with a 
relatively narrow focus), the simulation should build 
toward program management (several or a large number of 
projects with broad implications and focus). 

(3) The scenario should be built around one of three familiar 

educational settings: a university setting, a regional 

laboratory/R S- D Center/State Department setting, or a 
public school setting. The decision was made that a 
setting which might combine a regional laboratory and 

an R & D Center with emphasis on the former would be 
used for the development of the materials. 

(4) In connection with the above selected scenario, the following 

items would have to be included or dealt with: the organiza- 

tional structure Including sub-units, the statistical and 
dupl icat ional limitations of the setting, the varied per- 
sonnel and the background of the organization. 
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(5) The simulation exercise would permit and require application 

of the following concepts which were treated by previous 
staff presentations: project definition, networking, sched- 

uling,* resource allocation problems, updating, and problem 
Identification* 

(6) The consideration of the specific problem/project to be 
used in the materials was narrowed down to those with 
which the staff designing the simulation exercise were 
familiar* They included computer assisted instruction, 
development of non-graded school, the management of a 
recently approved proposal, or the response to a "request 
for proposal" (RFP) from a Federal agency* The latter of 
these four was ultimately chosen as the one which would 
be used* 

(7) It was further decided that the following items needed 

to be developed: a statement of program focus;, a state- 

ment on the history of the hypothetical organization; a 
description of the organization or organizational chart; 

a statement regarding the number, size, and qualifications 
of the staff; a description of available facilities; 
statements of relationship to USOE, state departments of 
education, universities and colleges, R & 0 centers, and 
public schools* 
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(8) The participants were to produce a work breakdown structure 
and network which was to be scheduled and later replanned 
for the same project, 

(9) The project was to be of their own choosing in response to 
the RFP. They were to be working In a simulated hypothetical 
organization in groups which require some role-playing. 

Figure I gives a description of the inputs given to the participants 
under the three major topics and the expected outputs for the initial 
simulation exercise. 

It can be seen from Figure 1 that the original simulation exercise was 
designed to take place in three sessions. Each of these was to require 
about one-half day of the second, third, and fourth days of a five day 
training session. From the inputs which the participants received, they 
were required to spend the first session analyzing the materials and 
defining the project in a proposed response to the RFP, For this to be 
accomplished successfully the participants had to assume that they were 
part of the organization which was responding to this RFP and make a 
relatively large number of decisions involving a number of complex 
management functions. These included: communications to the point of 

agreement and action; selection of appropriate information from among 
that which was given; identification of major goals and a hierarchical 
breakdown of the functions, products, and work to be done; arrangement 
of the defined tasks or activities Into a sequentially ordered network 
showing interrelationships. The complexities of these management problems 
were such that this was not accomplished in one session* therefore they 
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were really completed during the second simulation session. As the 
simulation materia 1£ were revised over the past year (they have been used 
in seven training programs, and we presently have data for six of these 
programs) we have alloted more total time but In shorter sessions to the 
simulation activities. At present, the simulation constitutes about half 
of the total training time. 

As can be seen from Figure I, the topics for the second and third 
sessions were involved with other management planning and control functions 
and required different inputs and outputs. Although these will not be 
discussed here, it should be noted that success in these sessions depended 
upon the efforts and success in achieving the product for the first 
session, and sessions two and three were based on these previous efforts. 

During the simulation process the participants were seated at 
work tables in groups of from four to six. Each group represented 
middle level management positions In a separate aon-compet I ng hypothetical 
organization resembling a regional laboratory. The chairman of the group 
was arbitrarily assigned so as to facilitate quicker organization and 
action of the group. As work progressed, the additional input was given 
to the participants. This input; usual ly given in the form of a memorandum, 
required additional management decisions and other functions typical of 
an R & 0 project manager. 

After about 8-9 hours of the simulation process, a board of trustees 
meeting of the organization was called at which time each group summarized 
its plan, problems, and work accomplished. This provided a vehicle for 
a group summary of management problems, difficulties of operation, and 
experience in presentation of a proposal to a top level management group. 
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At the end of the training program the participants were ashed 
evaluate the simulation exercise as a part of the total program, 

III. EVALUATION OF SIMULATION MATERIALS AND USE 

Evaluation of the simulation materials was provided by a form using 
ten open-ended questions (See appendix #1), The purpose for obtaining 
this type of feedback was to provide information and suggestions for 
Improving the simulation materials. Minor differences were made In the 
use and/or content of the exercise as the development of the materials 
continued. For example, on the basis of the first session where the 
simulation was used, it was determined that the RFP was too complex and 
obtuse for analysis in the available time. Therefore, another RFP was 
used and found more adaptable to the simulation objectives. 

The evaluation form was basically the same for five of the six training 
sessions in which the simulation was used. For these five training sessions 
the open-end responses were examined and classified in one of seven ways. 

If the response was totally positive by such comments as "very good," "good," 
"well done," or some phrase which was obviously positive in nature, it was 
rated as pos i t i ve. In similar manner, if the comment was negative in tone 
by such comment as "poor," "inadequate," or a phrase such as "little feedback, 
"not enough time," etc., the response was rated as neqat i ve « if the comment 
written involved both positive and negative statements, or was of an indeter- 
minate nature such as "ok," "adequate," or "fair" it was rated as neutral , 
unless in the context of the part icipanil’s paper, that comment was probably 
either positively or negatively intended. 
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A reaction was rated as suggestion If the participant made a suggestion 
without giving a value judgement to it. If however, a suggestion was made 
after a comment such as M fine but could have ... n or "poorly done, but 
could be Improved by . . ." then the reaction was rated either positive- 
plus suggestion or negative-plus suggestion. Finally, If the Item was 
left blank on the evaluation sheet, the rating of no-response was given. 

Table I lists the reactions obtained for the open-end evaluation 
form for the session. It should be noted that such a rating of reactions 
for questions dealing with suggestions for improvement and other general 
comments may have little meaning. Because of a change In the form Itself 
after it was used two times, there Is no data provided for question 10 for 
two training sessions, the AERA Presession and session number 1. On the 

basis of the data presented In Table I, the following comments can be 
made. 

(1) The most positive reactions were toward the realism of the 
simulation exercise (Item 2) and the correlation of the sim- 
ulation sessions with their preceding Instructional sessions 
(Item 5). The third and fourth most positive parts were the 
administration and organization of the simulation exercise 
(Item l) and the information contained In the materials (Item 4). 

(2) The most negative reactions were to the time length for each 
session (Item 3) (over 1/3 of these came from the AERA Presession), 
the explication of the roles to be played and the value of role 
playing (item $), and the feedback from the staff (Item 7)« 



( 3 ) 
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The item which had an approximately equal positive and negative 
reaction referred to the clarity of the end products (item 8). 

It is suggested that these responses might have been different 
if the question were changed so that It referred to either the 
clarity seen before the work was undertaken or the clarity of 
the products achieved at the end of each session. 

(4) The largest number of neutral responses were obtained on the 
items concerning the time length of each session (item 3) > the 
information in the materials (Item 4) and the clarity of the 
end product (Item 8). 

(5) The largest number of non-value- laden suggestions were given 

In response to the Items asking for suggestions for Improvement 
(Item 9), as would be expected, end followed by the items 
referring to administration and organization (Item l) and 
feedback from the staff (Item 7)# 

For the sixth session, the workshop coordinator constructed a semantic 
differential scale consisting of six concepts, with ten pairs of terms 
relating to each concept. One of these concepts was "simulation activities," 
On this scale there*- were no negative ratings, an average of one neutral 
rating per pair, and an average of seventeen persons rating the item posi- 
tively. These positive ratings were about equally split between the most 
positive response and the most neutral response. 

On a separate critique form an Item asked which features of the 
training program (or workshop) "were especially facllltative to (learning)". 
The responses by 18 participants showed the simulation to have the strongest 
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effect (N-12), followed by parts of the lectures (N*?) and visuals (N=l). 

Therefore It can be seen that the participant evaluation of the simula- 
tion was very positive, with no negative reaction for this particular 
workshop. However, It could be argued that If the same open-ended form 
had been used, the responses would have included suggestions for improvement 
anc^ by Impl icat Ion, at least some negative responses. 



SUMMARY 



The purpose of this paper was to present a discussion of the. develop- 
ment and use of simulation in the training of R & D project managers. 

The need for a realistic, broad-based exercise to train R & D project 
and program managers in education led to the exploratory development and 
use of a simulation exercise. This exercise, now comprising about half of 
a one-week training program, was designed and tested over the past 15 months 
with almost 200 participants, 

A summary of the decisions made regarding the design, development, and 
use of the simulation materials was presented. This was followed by a 
discussion of the open-ended evaluation form used for five of the seven 
training programs where the simulation has been used. Useable feedback 
from 138 participants was rated in one of seven ways as a basis for a 
general evaluation of the instrument. An additional 18 participants provided 
an evaluation by means of a semantic differential scale. 

The evaluations showed overall positive reactions toward the use of 
simulation in the training program. The problems of the time length of the 
simulation sessions; of the place and amount of role playing; and the amount 
and appropriateness of feedback are currently the subject of further study. 
Much more developmental and experimental work needs to be done. At this time, 
however, it can be said that simulation appears to be a very promising tool 
in the training of R & D project managers. 
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appendix #i 



Educational Research Management Center 
College of Education 
The Ohio State University 
Columbus, Ohio 43210 



MANAGEMENT GAME EVALUATION 



The SWINDLE-SIS management game in which you participated 
is in the process of being developed as a management instruc- 
tional device. In order to revise the game, we would like 
to have your comments and suggestions regarding the several 
points listed below. Use the reverse side to make any more 
general comments or suggestions you might have. 

1. Administration and organization of the game: 



2. Realism of the game: 



. Time length for each session of the game: 



4. Information contained in organization description and 
action memorandums: 



5. Correlation with instructional sessions preceding game play: 



6. Explication of roles to be played. Is it realistic to 
attempt to play roles? If not, could anything be 
substituted to require adaptation to the context of the 
si tuat ion? 



7. Feedback from staff regarding your group's actions 

during sessions or at end of sessions. Suggestions as 
to how this could be improved: 



8. Clarity of end products to come out of each session: 



9. How could the game be improved? (Use reverse side as 
needed) : 



10. Other or general comments: 



6/25/68 
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